Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update installation instructions #219

Merged

Conversation

andreasWallner
Copy link
Collaborator

Major changes are to update JDK (17) and scala instructions (via Coursier as now recommended).
Also recommend oss-cad-suite over self-compiled tools.

@Readon
Copy link
Collaborator

Readon commented Oct 9, 2023

good, this also reminds me that the documentation for docker installation is also required.

@andreasWallner
Copy link
Collaborator Author

@Readon Do you have oss-cad-suite verilator running on Windows by any chance?
With the current instructions I end up with a DLL where java is unable to find the symbols for simulation in the end...

@Readon
Copy link
Collaborator

Readon commented Oct 9, 2023

@Readon Do you have oss-cad-suite verilator running on Windows by any chance? With the current instructions I end up with a DLL where java is unable to find the symbols for simulation in the end...

I have not use oss-cad-suite for a long time, I could test it later.

@Dolu1990
Copy link
Member

Dolu1990 commented Oct 9, 2023

Thanks for the update :)

@andreasWallner
Copy link
Collaborator Author

@Dolu1990 I'd hoped it would be that easy, still have to debug the Windows portion though...

@Readon
Copy link
Collaborator

Readon commented Oct 10, 2023

@Dolu1990 I'd hoped it would be that easy, still have to debug the Windows portion though...

It might be a better idea to use docker as a better and easy maintainable method.

@Readon
Copy link
Collaborator

Readon commented Oct 10, 2023

@Readon Do you have oss-cad-suite verilator running on Windows by any chance? With the current instructions I end up with a DLL where java is unable to find the symbols for simulation in the end...

Btw, we should have noticed that the OSS-CAD is a rolling update version. I have tested previous version at 20220427 version, it seems still works.

@andreasWallner
Copy link
Collaborator Author

@Readon while I see some advantages with using the container (I have done so for a while at work for different projects) I'm not sure I'd recommend this generally: You still need WSL on windows, it seems to lead to easy confusion about which file system one is in (as exhibited by some of my colleagues), performance is worse than in WSL proper and you'd need to run your IDE in the container (which currently only VSCode supports if you don't mess around with X forwarding?) to get all features

@Readon
Copy link
Collaborator

Readon commented Oct 12, 2023

@Readon while I see some advantages with using the container (I have done so for a while at work for different projects) I'm not sure I'd recommend this generally: You still need WSL on windows, it seems to lead to easy confusion about which file system one is in (as exhibited by some of my colleagues), performance is worse than in WSL proper and you'd need to run your IDE in the container (which currently only VSCode supports if you don't mess around with X forwarding?) to get all features

Yes,a large WSL might be a problem. However, there is a VSCode extension called devcontainer which make it simple and easy. Normally, you can just use VSCode on Windows's file system. The speed of mainstream lib compilation and test is faster then msys2 as I tested before.

@andreasWallner andreasWallner force-pushed the update_install_steps branch 2 times, most recently from 9870386 to 9de1997 Compare October 21, 2023 13:24
@andreasWallner andreasWallner marked this pull request as ready for review October 21, 2023 13:29
@andreasWallner
Copy link
Collaborator Author

I'm still stuck on the freeze that I get on the native Windows install, therefore I added the note to use WSL so that we can at least get the updated Linux instructions and some update on Windows.

Also added some info for the container but I didn't really want to go into how to install/use docker/podman as that is a huge rabbit hole I think we shouldn't get into since all of that is documented somewhere else.

Ad freeze: simulation works, but it seems Java just freezes after the JNI for verilator finishes execution. I'll look at this again, but again - don't want to keep the other stuff outdated while trying stuff...

@Readon
Copy link
Collaborator

Readon commented Oct 21, 2023

I'm still stuck on the freeze that I get on the native Windows install, therefore I added the note to use WSL so that we can at least get the updated Linux instructions and some update on Windows.

Also added some info for the container but I didn't really want to go into how to install/use docker/podman as that is a huge rabbit hole I think we shouldn't get into since all of that is documented somewhere else.

Ad freeze: simulation works, but it seems Java just freezes after the JNI for verilator finishes execution. I'll look at this again, but again - don't want to keep the other stuff outdated while trying stuff...

There is another possible way to use SpinalHDL on whole msys2 environment.

  1. install jvm & sbt by sdkman, which provide msys2 compatible version, such as JDK 17 from Microsoft. This version works on my laptop.
  2. intall verilator at version 4.228.
  3. package & install yosys and sby into msys2 packages.
  4. install z3 & yices from msys2 mainstream repos.

@andreasWallner
Copy link
Collaborator Author

I will give this a try, thanks!
A few things:

  • I can run simulations, it's that somewhere later the JVM get's stuck - @Dolu1990 seems to have had a similar problem in the past, which went away with a new install/computer...
  • What makes you say that OpenJDK17 from MS is MSYS2 compatible? I can't find any claim by MS that they would try to ensure that.
  • From what I've seen so far verilator version does not matter, but I'll go back and check again
  • Did you install verilator via MSYS2?
  • Are you using gcc from MSYS2?
  • If possible I'd like to avoid yet another tool, especially one that is not officially supported by the scala guys
  • I'm not willing to invest time in creating any mysy2 packages

If you want to update the instructions with your setup feel free to do so, otherwise I'll try to get a setup running in my VM (to ensure that I'm not missing something because I've had it installed for ages).

Would it be ok for you to merge as it is at the moment to have updated instructions until we can figure out some that work for all Windows machines?

@Readon
Copy link
Collaborator

Readon commented Oct 22, 2023

  • What makes you say that OpenJDK17 from MS is MSYS2 compatible? I can't find any claim by MS that they would try to ensure that.
    Previously I got JVM stuck while running sbt. When I switched to that version it will works good. But some other version might fail.
  • From what I've seen so far verilator version does not matter, but I'll go back and check again
    I just test the verilator 5.016 with SpinalHDL 1.9.3 it report some sv file missing.
  • Did you install verilator via MSYS2?
  • Are you using gcc from MSYS2?
    Yes, I just want to use a standalone MSYS2 environment, so that it is portable.
  • If possible I'd like to avoid yet another tool, especially one that is not officially supported by the scala guys
  • I'm not willing to invest time in creating any mysy2 packages
    I have done it before, the packages with fixed version can be provided on somewhere.
    However msys2 official repos only maintains packages they use, we could not get yosys and sby the missing packages merged in .

@andreasWallner
Copy link
Collaborator Author

thanks for the infos, I'll give the other JDK a try

I also got an error due to an sv file for some builtins - after setting VERILOG_ROOT correctly that worked for me

@Readon
Copy link
Collaborator

Readon commented Oct 26, 2023

@andreasWallner I create a release here through modifying msys2 installer for an all in one package. suggestion is welcome.

@andreasWallner
Copy link
Collaborator Author

andreasWallner commented Oct 26, 2023

@Readon Thank! Turns out that the JDK build didn't matter, but it seems that the code that verilator 5 generates is somehow an issue. Didn't expect that...

I'm now pointing at your build but since the only really special thing there are the yosys/sby packages I wanted to keep the description on what to install.
Only point there is that when I download the x86_64 packages from your release I can't install them:

$ curl -O https://github.com/Readon/MINGW-SpinalHDL/releases/download/v0.4.9/mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

$ pacman -U mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst
loading packages...
error: missing package metadata in mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst
error: 'mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst': invalid or corrupted package

Any idea?

@Readon @Dolu1990
As it stands this setup on Windows seems to be a bit finnicky so I'd like to keep the note w.r.t. WSL - I think it is a much simpler setup (and due to some past experiences mingw/msys being quite prone to breaking other stuff I like to avoid those two.)

I'd like to keep sbt installation via coursier since that is the officially recommended method, and it seems to work.
I also kept Adoptium since MSFT just joined the Adoptium foundation and it's expected that they won't update anymore - and it turns out it was not the reason for the hang.

I removed to oss-cad-suite for windows for now since I couldn't get that to work, independent of which JDK I used (some reference to another dll ends up in the simulator dll that java can't load it, but it's hard to find since DependencyWalker can't deal with some of the MSFT dlls and finds some infinite loop in the resulting graph, error during simulation is [error] Exception in thread "main" java.lang.UnsatisfiedLinkError: C:\Users\User\Downloads\SpinalTemplateSbt-master\simWorkspace\MyTopLevel\verilator\verilator_1.dll: The specified procedure could not be found)

I'd say that if we can fix the install issue the description should be good to go.

@Readon
Copy link
Collaborator

Readon commented Oct 26, 2023

@Readon Thank! Turns out that the JDK build didn't matter, but it seems that the code that verilator 5 generates is somehow an issue. Didn't expect that...

I'm now pointing at your build but since the only really special thing there are the yosys/sby packages I wanted to keep the description on what to install. Only point there is that when I download the x86_64 packages from your release I can't install them:

$ curl -O https://github.com/Readon/MINGW-SpinalHDL/releases/download/v0.4.9/mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

$ pacman -U mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst
loading packages...
error: missing package metadata in mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst
error: 'mingw-w64-x86_64-yosys-0.31-1-any.pkg.tar.zst': invalid or corrupted package

Any idea?

@Readon @Dolu1990 As it stands this setup on Windows seems to be a bit finnicky so I'd like to keep the note w.r.t. WSL - I think it is a much simpler setup (and due to some past experiences mingw/msys being quite prone to breaking other stuff I like to avoid those two.)

I'd like to keep sbt installation via coursier since that is the officially recommended method, and it seems to work. I also kept Adoptium since MSFT just joined the Adoptium foundation and it's expected that they won't update anymore - and it turns out it was not the reason for the hang.

I removed to oss-cad-suite for windows for now since I couldn't get that to work, independent of which JDK I used (some reference to another dll ends up in the simulator dll that java can't load it, but it's hard to find since DependencyWalker can't deal with some of the MSFT dlls and finds some infinite loop in the resulting graph, error during simulation is [error] Exception in thread "main" java.lang.UnsatisfiedLinkError: C:\Users\User\Downloads\SpinalTemplateSbt-master\simWorkspace\MyTopLevel\verilator\verilator_1.dll: The specified procedure could not be found)

I'd say that if we can fix the install issue the description should be good to go.

you did not actually download the file. it's still empty. the all in one build is only for ones that wish to use an offline environment.

@Readon
Copy link
Collaborator

Readon commented Oct 26, 2023

wget might works.

@Readon
Copy link
Collaborator

Readon commented Oct 26, 2023

use sdkman to manage java related thing is just want to keep the msys2 portable. it can be set to have its own PATH and related env variables, without inherit them from Windows.

@andreasWallner
Copy link
Collaborator Author

you did not actually download the file. it's still empty. the all in one build is only for ones that wish to use an offline environment.

thanks, should have cought that - fixed the curl call, added python packages missing in default installation needed for formal - went through the whole thing again and it works now.

@Readon
Copy link
Collaborator

Readon commented Oct 27, 2023

you did not actually download the file. it's still empty. the all in one build is only for ones that wish to use an offline environment.

thanks, should have cought that - fixed the curl call, added python packages missing in default installation needed for formal - went through the whole thing again and it works now.

I am also check the bugs of existing packages. It would be done soon.

@Readon
Copy link
Collaborator

Readon commented Oct 27, 2023

Overall, If user can access internet, then use packages in MINGW-Spinal repos is enough. They can install whatever they want step by step. If they do not have internet, or just want to download and use, then MSys-installer is a good choice.

@Readon
Copy link
Collaborator

Readon commented Oct 27, 2023

According to your comment, I have also tested the temuri jdk from adoptium, it works also. I have switched the jdk to that.

@Readon
Copy link
Collaborator

Readon commented Oct 27, 2023

The verilator's version should be downgraded to 4.228 before 1.9.3 (include).

@andreasWallner
Copy link
Collaborator Author

The verilator's version should be downgraded to 4.228 before 1.9.3 (include).

I don't understand what you mean, in the current description I do state that 4.228 is the latest supported, and the instructions manually download and install that (since it couldn't be installed here via pacman -S mingw-w64-x86_64-verilator=4.228-1)

@Readon
Copy link
Collaborator

Readon commented Oct 27, 2023

The verilator's version should be downgraded to 4.228 before 1.9.3 (include).

I don't understand what you mean, in the current description I do state that 4.228 is the latest supported, and the instructions manually download and install that (since it couldn't be installed here via pacman -S mingw-w64-x86_64-verilator=4.228-1)

Sorry for the misleading, don't bother on that. I just give a note on the modification of the msys-installer for spinal.

@andreasWallner
Copy link
Collaborator Author

Understood, do you think we can merge this as it is?

@Dolu1990
Copy link
Member

One reason why i was avoiding precompiled binaries date from long ago in the dark ages of open-source hardware, where i had to pick the blessed version of tools to avoid bugs, and not waist time.

I don't know how much it is still the case today

WSL sound good to me,

@Readon
Copy link
Collaborator

Readon commented Oct 27, 2023

One reason why i was avoiding precompiled binaries date from long ago in the dark ages of open-source hardware, where i had to pick the blessed version of tools to avoid bugs, and not waist time.

I don't know how much it is still the case today

WSL sound good to me,

haha, I think if we can freeze on some version, then less effert is required. That's why I want to build an all in one version.

@Readon
Copy link
Collaborator

Readon commented Oct 28, 2023

also, the yosys and symbiyosys in existing msys repos use a wrong path, which fails to run yosys-smtbmc. there are too much to maintain, so that a frozen release is a good alternatives.
oss-cad-suite are also fails by rolling release policy.

@andreasWallner
Copy link
Collaborator Author

I'll take my impression that there are no changes you guys proposed as an OK to merge this - if not please say so and I'll happily fix the stuff.

@andreasWallner andreasWallner merged commit b88c0a1 into SpinalHDL:master Oct 28, 2023
3 checks passed
@andreasWallner andreasWallner deleted the update_install_steps branch October 28, 2023 13:20
@Dolu1990
Copy link
Member

Thanks :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants